To address this problem, existing methods partition the overall DPDP into fixed-size sub-problems by caching online generated orders and solve each sub-problem, or on this basis to utilize the predicted future orders to optimize each sub-problem further. However, the solution quality and efficiency of these methods are unsatisfactory, especially when the problem scale is very large.

Specifically, we randomly sample an example and adopt a first-order procedure to approximate the Hessian/vector product, which makes computing more efficient by interpolating two neighboring gradients.

Sparse training has received an upsurging interest in machine learning due to its tantalizing saving potential for the entire training process as well as inference.

Di Lin is the corresponding author of this paper.

Then momentum acceleration is also introduced into our dual accelerating update step.

Then momentum acceleration is also introduced into our dual accelerating update step.

Surprisingly, we find that most of the studied methods yield similar results.